Sequential Model Averaging for High Dimensional Linear Regression Models

In high dimensional data analysis, we propose a sequential model averaging (SMA) method to make accurate and stable predictions. Specifically, we introduce a hybrid approach that combines a sequential screening process with a model averaging algorithm, where the weight of each model is determined by its Bayesian information (BIC) score (Schwarz, 1978; Chen and Chen, 2008). The sequential technique makes SMA computationally feasible with high dimensional data, because the averaging process assures the prediction’s accuracy and stability. Theoretical results show that SMA not only yields a good model, but also mitigates overfitting. In addition, we demonstrate that SMA provides consistent estimators for the regression coefficients and yields reliable predictions under mild conditions. Both simulations and empirical examples are presented to illustrate the usefulness of the proposed method.